CDK9 is an essential drug target correlated to the development of acute myeloid leukemia (AML). Starting from the hit compound 10, which was discovered through a screening of our in-house compound library, the structural modifications were carried out based on the bioisosterism and scaffold hopping strategies. Consequently, compound 37 displayed the optimal CDK9 inhibitory activity with an IC50 value of 5.41 nM, which was nearly 1500-fold higher than compound 10. In addition, compound 37 exhibited significant antiproliferative activity in broad cancer cell lines. Further investigation of in vivo properties demonstrated that compound 37 could be orally administrated with an acceptable bioavailability (F = 33.7%). In MV-4-11 subcutaneous xenograft mouse model, compound 37 (7.5 mg/kg) could significantly suppress the tumor progression with a T/C value of 27.80%. Compound 37 represents a promising lead compound for the development of a novel class of CDK9 inhibitors for the treatment of acute myeloid leukemia.